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APPROXIMATION  FOR  BAYESIAN  ABILITY  ESTIMATION 


KEY  WORDS:  Ability  estimation,  Bayesian  approximation,  binary 
variables,  item  response,  two-parameter  logistic. 

Abstract 

An  approximation  is  proposed  for  the  posterior  mean  and  stand¬ 
ard  deviation  of  the  ability  parameter  in  an  item  response  model. 
The  procedure  assumes  that  approximations  to  the  posterior  mean 
and  covariance  matrix  of  item  parameters  are  available.  It  is 
based  on  the  posterior  mean  of  a  Taylor  series  approximation  to 
the  posterior  mean  conditional  on  the  item  parameters.  The  method 
is  illustrated  for  the  two-parameter  logistic  model  using  data 
from  an  ACT  math  test  with  39  items.  A  numerical  comparison  with 
the  empirical  Bayes  method  shows  that  the  point  estimates  are  very 
similar  but  the  standard  deviations  under  empirical  Bayes  are  about 
two  percent  smaller  than  those  under  Bayes.  The  effect  of  sample 
size  is  demonstrated  by  illustrating  the  increase  in  the  standard 
deviations  for  a  smaller  data  set. 
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Ability  Estimation 


We  consider  estimating  the  ability  of  an  individual  based  on 
the  person's  dichotomous  responses  to  a  set  of  test  items  whose 
characteristics  are  partially  known  through  responses  to  the  same 
set  from  other  individuals  belonging  to  the  same  population.  Each 
item  is  characterized  by  an  item  response  function  which  defines 
the  probability  of  a  correct  response  to  the.  item  by  an  individual 
with  ability  8.  We  assume  such  a  function  to  be  known  except  for 
some  parameter  t,  and  use  the:  notation  P(  y  |  6 ,  £ )  to  represent  the 
probability  of  correct  response  (y=l)  or  incorrect  response  (y=0) 
by  an  individual  with  real  valued  ability  6  to  an  item  with  para¬ 
meter  1 .  An  example  of  such  a  function  is  the  two-parameter  logistic 
model  defined  by 


p  (y  I  e ,  -x , s ) 


explya  (‘"-8)  } 
1+expia (8-8 ) } 


(  1  ) 


y  =  0,1,  where  -®  <  0  <  <*>,  •;  =  (  t,i?)  is  two  dimensional  with  x  >  0 

and  -°°  <  8  <  °°. 


We  assume  the  calibration  of  the  test  is  based  on  responses 
to  k  items  by  a  group  of  n  individuals  representative  of  some 
target  population  for  which  abilities  are  to  be  measured.  We  let 
y  =  0  or  1  according  as  the  response  by  examinee  i  to  item  j  is 
incorrect  or  correct.  We  assume  conditional  independence  among  the 
responses  so  that  the  joint  probability  of  the  nxk  matrix  y  of 


responses  y^  is  given  by 


Ab 1 1 i t  v  E 
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c  t in.it  ion 


where  M  =  (0  , . . . , 0^ )  represents  the  abilities  of  the  n  individuals 
and  \  =  ( - 1  >  •  •  •  *  the  parameters  of  the  k  items. 

One  standard  method  used  in  calibration  is  to  obtain  the 

a  A 

joint  maximum  likelihood  estimate  (a  e>.  )  of  (0,£)  using  a  procedure 
such  as  LOGIST  (Wingersky,  Barton,  and  Lord,  1982).  The  cali¬ 
brated  items  are  then  used  to  measure  the  ability  a  for  a  new 
individual  with  item  responses  x  =  (x^,...,x^)  by  finding  the  value 
of  r5  which  maximizes  the  probability 


P  (X  !  «,  %  )  =  JIP(x  .  I  A,C  . 
JL  j  3  L3 


This  procedure  is  straight  forward  and  relatively  easy  to  implement, 
however  it  is  known  to  be  biased  outwards  for  extreme  values  of  0 
(Lord,  1983a),  is  subject  to  an  occasional  nonexistence  of  a  solu¬ 
tion,  and  fails  to  provide  a  good  measure  of  uncertainly  in  the 
estimated  0 

A  related  empirical  Bayes  procedure  is  to  first  assume  a 
prior  distribution  tt  of  a  and  estimate  ",  by  marginal  maximum 
likelihood.  Then  derive  the  posterior  mean  of  O  conditionally  on 
assumed  to  equal  the  marginal  maximum  likelihood  estimate. 

The  resulting  estimate  of  0  is  an  empirical  Bayes  (Cox  and 
Hinkley,  1974)  estimate  and  has  been  demonstrated  on  different  models 
by  Bock  and  Aitkin  (1981)  and  Rigdon  and  Tsutakawa  (1983) ,  among 
others . 

The  Bayesian  approach,  which  we  adopt  here,  assumes  prior 
distributions  on  both  0  and  i,  and  uses  an  approximate  posterior 
mean  and  variance  of  0  tc  make  inferences  regarding  the  unknown 
9.  The  approximation  for  the  posterior  mean  E(P|X)  is  based 
the  posterior  expectation  of  a  Taylor  series  approximation  of  the 
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Ability  Estimation 

conditional  posterior  expectation  E(0|x,£).  When  the  posterior 
distribution  of  £  is  normal,  it  reduces  to  Lindley's  (1980) 
approximation  whose  general  form  requires  the  computation  of  a 
large  number  of  3rd  partial  derivatives  of  the  loglikelihood 
function.  We  will  first  discuss  our  method  when  the  individual 
whose  0  is  being  measured,  is  a  member  of  the  calibrating  set, 
e.g.  6  =  9^.  We  then  show  how  the  same  approximation  can  be 
modified  and  used  on  a  new  individual  from  the  same  target  pop¬ 
ulation  . 


Approximate  Posterior  Mean  and  Variance  of  6 


Assume  that  0^,  .  .  9  are  independent  and  identically 

distributed  (iid)  according  to  some  known  prior  p(0).  Also  assume 
that  are  independently  distributed  according  to  some  known 

priors  p  (  p  (  )  and  independent  of  9^,.-..  ,  0  .  Then  the 

marginal  posterior  pdfs  of  £  and  0  are  given  by 


and 


r 


P (9 | y) 


(5) 


As  shown  in  Tsutakawa  and  Lin  '1986)  the  EM  algorithm  can  be  used 

A 

to  compute  the  posterior  mode  t  of  £  and  the  posterior  covariance  of 
•'  can  be  approximated  by  Z,  the  inverse  of  the  negative  Hessian  of 

A 

the  log  posterior  evaluated  at  £ .  Due  to  the  symmetry  between  (4) 
and  (5),  it  may  appear  that  similar  methods  may  be  adopted  to 
compute  the  posterior  mode  of  0.  However,  the  analogous  approach 
presents  serious  numerical  problems  since  the  integrals  in  (5) 
are  multiple  intergrals  in  contrast  to  the  single  integrals  in 
(4). 

Instead  we  shall  try  to  derive  the  posterior  expectation 
of  0  by  approximating  tie  integral  in  the  expression, 
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where  E(0|y,£)  is  the  posterior  expectation  of  6  conditionally 

on  £,  which  is  used  by  the  empirical  Bayes  approach  upon  substituting 

£  with  its  marginal  maximum  likelihood  estimate. 

Let  w(£)  =  E  | y^r  £)  and  let  w,  wr,  w^  and  wrgt  denote 

the  values  of  w(£)  and  its  first  three  derivatives  all  evaluated 

A 

at  £.  Then  according  to  Lindley  (1980),  under  regularity  conditions 
(6)  may  oe  approximated  to  order  0(n  ^  )  by 


W  +  ijZw  T  +  h  l  A  ,w  T  T 

2  rs  rs  rst  u  rs  tu 


where  Arst  is  the  third  partial  of  the  log  posterior  evaluated  at 

A 

£  =  £  and  t  are  the;  elements  of  I.  The  last  term  accounts  fcr 
~  ~  rs 

the  skewness  of  the:  posterior  distribution  of  £  and  if,  in  particular, 


the  posterior  is  normal,  A  vanish( Anderson ,  1958,  p.  39)  and  (7) 

IT  St 


reduces  to 


m  “  w  +  *  Ew  T  . 
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A 

If  p (C| y)  is  normal  with  mean  £  and  covariance  matrix  (( °rs)) , 
then  the  third  moments  of  £  vanish  and 


r 

j  w(£ )  p  (£  |  y )  d£ 


=  w  +  %  1  w 


a 

rs  rs 


(  10) 


Upon  replacing  (  (°rs)  )  by  its  estimate  (  (Trs) )  we  have  (8). 

Mosteller  and  Wallace  (1964,  p.  151)  refer  to  (10)  as  the 

standard  approximation  for  the  case  of  known  means  and  covariances 

A 

and  point  out  the  need  to  consider  the  bias  £  ~  E(£|y) 

and  third  derivatives  when  the  distribution  is  nonsymmetric . 

The  posterior  variances  and  covariances  of  6  may  be  similarly 

approximated  by  first  obtaining  an  approximation  to  E(9^0j|y)  by 

repeating  (8)  with  w(£)  =  E  (0  ^  J,  y.^ , y^  ,  £  )  and  then  making  the 

appropriate  substitutions  in  E(9^0_.|y)  -  E  ( 0 ^ | y ) E ( 6 .  | y ) .  The 

2 

approximate  variances,  s  t  and  covariances  can  be  helpful  m 
assessing  the  uncertainty  of  a  particular  individual's  ability  or 
in  comparing  the  abilities  of  two  individuals. 

Suppose  we  now  have  the  n+lst  or  a  new  individual,  with 
unknown  9,  whose  response  to  the  same  k  items  is  x  =  (x^, . . . ,x^) • 

We  can  repeat  the  entire  procedure  including  the  updating  of 

A 

(£,I)  after  replacing  y  with  (y,x).  However  this  would  be  quite 
costly  for  routine  evaluations.  It  would  be  much  simpler  retaining 

A 

(£,£)  from  the  first  n  individuals  and  using  x  in  w(£)  =  E(6|x,£) 

2 

only.  When  n  is  moderately  large,  the  change  in  (m,s  )  through  the 

A 

updated  (£,£)  by  the  addition  of  a  single  individual  would  be 

quite  negligible  relative  to  the  variability  in  E(9|x,£)  or 
2 

E  ( 0  |x,£)  due  to  different  values  of  x. 
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Computational  Details  for  the  Two-parameter 
Logistic  Model 


In  this  section  we  present  computational  expressions  for  the 
evaluation  of  w  and  wrs  in  the  case  of  the  two-parameter  logistic  model 

A 

(1)  with  a  N(0,1)  prior  on  0.  A  derivation  of  (£,I)  based  on  the 
EM  algorithm  can  be  found  in  Tsutakawa  and  Lin  (1986). 

For  this  model  we  have 

w(|)  =  E(0i|Y.,O 

=  9iP(0ilyi»  ^dei'  (ii) 


where 

p  (9i  I  yi'  V 

and 

p(£>)  =  exp  (-"2/2)  ,  <  0  < 

/  2  rr 

Following  the  notation  in  Tsutakawa  (1984), for  each  i  =  1 . n  and 

u  =  1 , . . . , k  let 


P(U  11  P(Yj  -i  I 
=  1  j~l  1  3 

!plVjfi  p'Lj'V'W 


g  1  ( i ,  u ,  0  )  = 


3P(y.  |  0  , ot  ,3  ) 

2 iu 1  u  u 

TK 


/P(yiu!» 


t  ,3 

U  U 


12) 
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g2 ( i , u, o )  = 


3P(y.  I  0,a,  ,6  ) 


1U‘  u  u 


c  b 


hl;L(i,u,(?)  = 


3  P(y.  |  9 ,  ot  ,13  ) 


—  2-  u  u  /P{y<tJd,-\,,3',), 


jn. 


iu  1  u  u 


h  i  2  ( i ' u »  6  )  = 


3  p<yiuivvfSu) 

?,.i  33 

li  u 


•/P  (y  .  I  e  ,  a  ,6  )  , 
J1U  u  u 


h22(i,u,0)  = 


J2P(y.  ,3  ) 

1U  u  u 


-/P(^u!e'%'eu}' 


00u  =  a  +  exp[-otu(8  -  3u)J}‘  ,  and  v0u  =  1  -  4>0U- 


By  taking  derivatives  it  is  readily  shown  that 


“  <yiu  -  ♦.„>  e  -  v- 


g2(i,u,P)  = 


j  (y . 
u  iu 


V  u 


(  13  ) 


h,  .  (i,  U,  H  )  =  (y.  -  v*-  -  *>  ) 

XI  1  11  -Ml  *  1 1  •  •  U 


IU 


u  "U 


V2- 


h12(i,u,0)  *-(yiu  -  4>0u)  L  1  + 


u  ’:U  "gu 


V.,  >  (  6  -  6,.)  ] 


and 


h22(i,u,e)  -  (yiu  - 


(We  note  that  there  is  a  minus  sign  missing  in  the  expression  for 
h^2  in  Tsutakawa  (1984)).  Now  define  the  following  posterior 
expectations  of  the  derivatives  and  their  products  by 


i  J~r< 


Ability  Estimation 


10 


r00 

gs(i/U)  =  gs(i,u,0)p(6 | yi , £ ) , 


hst  (l,U)  =  j  hgt  (i,u,0)p(6  |yi,Ud-, 


r°° 

dgt  (i,u,v)  =  I  gs(i,u,0)gt  (i,v,e)p(e  |yi,^)d‘1. 


s,t  =  l,2,u,v  =  l,...,k,u  f  v,  where  p(0|y^»£)  is  defined  by  (11). 


Nov;  denote  the  second  derivatives  of  w  ( % )  evaluated  at 

A 

=  C  by 


Wll(U'V)  =  dw  (C)/3oiu3av, 


w12(u,v)  =  3w(C)/3au38v»  ( i 5 ) 

A 

w21(u,v)  =  3w(O/36u30iv, 


and 


w22(u,v)  =  3w(5)/35u)5v  . 


These  derivatives  may  be  computed  using  the  following  expressions. 


wst<u»u)  *  Ef0hst (i,u,9) }  +  { 2gg  (i , u) gfc (i , u)  -  hgt  ( i , u )  -  E ( - ) 
-  gg (i,u) E{0gfc  (i, u, 0) }  -  gfc ( i , u ) E{  -gs ( i , u ,  •'*  ) ; 

(lb) 

and,  for  u  ^  v, 


w  (u,v)  =  (  2gs  (i,  u)  gt  (i,  v)  -  dgfc  ( i ,  u ,  v)  ■  E  ( •' ) 


(  17  ) 
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E  ■  ■  -  g  (  1  ,  u ,  ■  ;  g  (  l  ,  v  ,  ) 


q  (  1 ,  v  )  E  j  i  l  ,  a  ,  I  -  q  i  ,  a  '  E  ■  ;  i  ,  v  , 

t  ^  5  t 


where  E  denotes  posterior  expectation  wit!  respect  to  the  dens  it1 
p  p  y  »  i  defined  by  (11)  when  = 

The  numerical  evaluation  of  wst*u,v^  requires  numerically 
integrating  a  number  ■'!  integrals  of  the  type 


'"Pi  v 

i  i  *  i :  1 


where  tie  function  hi  •  .  )  varies  from  one  integral  to  the  next.  The  use 
of  scaling  tor  improved  accuracy  in  tie  Gauss-Hermte  approximation 
is  discussed  in  Tsutakawa  (1984). 
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Example 

Our  method  will  now  be  illustrated  on  a  sample  data  set  from  an 
ACT  math  test  previously  used  by  Tsutakawa  and  Lin  (1986).  The: 

:  v  1  of  responses  on  k  =  39  items  by  n  =  4<:  respondents. 

sinu  f he i r  method,  {%,  I)  were  initially  recomputed.  The 
posterior  means  and  standard  deviations  (m,s)  were  then 

:  't  *  he  tost  100  responden  t  s  and  are  plotted  against  the 

A 

empirical  .r.*  •_  i  parts,  l  E  t  y,"  )  and  SD  <  '■  y ,  • 

a  . 

«:iere  is  r  n*-*  marginal  maximum  likelihood  estimate  of  £  ]  in  Figures  1 

ind  2.  We  note  that  the  means  under  the  two  procedures  are  almost 
:•  •'’-.  'il  bu*-  *■  he  standard  deviations  could  be  quite  different. 

(a.  *■  lUiiar,  posterior  standard  deviations  are  1.5  to  2.5 
percent  smaller  under  empirical  Bayes  than  approximate  Bayes. 

’‘xclanatem  for  this  is  that  the  empirical  Bayes  approach  does 

\ 

:v  t  account  for  the  uncert- imty  in  •'  (  Deelev  and  Lindley  1981'. 

A  plot  of  s  vs.  m  in  Figure  3  shows  that  the  larger  s  is  associated 
with  the  larger  m1,  suggesting  the  difficulty  of  estimatirg  extreme 
ability  values  based  on  the  giver,  test.  The  quadratic  relationship 
seen  m  the  plot  suggests  that,  for  routine  use,  s  may  be  expressed 
as  a  function  of  m  and  need  not  be  computed  for  all  individuals. 
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Effect  of  Sample  Size  n 

An  interesting  conjecture  involves  the  effect  of  the  sample 
size  n  on  our  Bayesian  estimates.  Although  the  posterior  moments 
of  ability  are  calculated  separately  for  each  individual  they 

A 

depend  on  the  values  of  (£,  Z  )  which  are  estimated  from  a  response 
matrix  which  may  contain  the  particular  individual's  response.  It 
is  natural  to  believe  that  as  the  size  of  the  response  matrix 
decreases,  the  posterior  standard  deviation  of  an  individual's 
ability  would  increase,  as  there  is  more  uncertainty  in  the 

A 

estimated  (  £,  I  )  . 

We  previously  found  ability  estimates  for  the  first  100 

A 

examinees  in  a  400  x  39  response  matrix  using  (  £,£  )  which  were 
derived  from  the  responses  of  all  400  examinees.  To  investigate 
our  conjecture  we  reestimated  the  posterior  moments  of  ability 
for  the  same  set  of  100  examinees,  using  (  £,I  )  derived  from 
the  first  100  examinees  only. 

The  initial  comparison  produced  an  interesting  result.  From 
individual  to  individual  it  was  found  that  the  posterior  standard 
deviation  of  ability  was  usually  smaller  than  the  corresponding 
standard  deviation  based  on  n  =  400.  We  do  not  feel  however  that 
our  initial  conjecture  is  untrue  as  the  following  discussion  shows. 

It  is  well  known  that  the  parameters  of  the  two-parameter 
logistic  model  are  not  unique.  The  item  parameters  which  were 
estimated  from  the  two  samples  are  quite  different  although  the 
items  are  exactly  the  same.  In  particular,  the  estimates  of  a  ^ 
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were  larger  for  n  =  100.  This  is  consistent  with  Lord  (1983b) 
who  shows  that  the  MLE  of  the  discrimination  parameter  is 

positively  biased,  particularly  for  small  n.  (Although  we  are  not  using 
MLE's  here,  our  prior  is  relatively  flat.)  Intuitively,  larger  values  of 

the  discrimination  parameter  will  tend  to  produce  smaller  posterior 
standard  deviations  of  ability.  We  feel  that  this  explains  why 

the  standard  deviations  did  not  increase  as  expected. 

In  order  to  put  both  sets  of  estimated  item  paramters  on 


a  common  scale  and  make  them  more  comparable,  we  propose  the  following 


transformation.  Define 


’’ 


k 

Z  8 

U  =  -  a  ^ - 1  ,  (19) 

k 


+  i 

a  .  =  ,  and 

3  s 


★ 

Sj  =  ^  +oB  j'  3  =  1 . k. 


The  corresponding  transformation  for  the  ability  parameter  is 

* 

9  ^  =u+  a9  ^ . 


(20) 
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One  reason  for  putting  the  restriction  on  the  item  parameter 
space  rather  than  on  the  abilities  is  because  the  item  parameters 
are  common  to  both  sets  while  the  ability  parameters  in  the  two 
sets  are  not  identical,  except  for  the  first  100.  Upon  applying 
the  rescaling  to  the  two  sets  separately,  the  rescaled 
posterior  standard  deviations  for  n  =  100  were  .6  to  14  percent 
larger  than  those  found  using  n  =  400.  This  can  be  seen  graphically 
in  Figure  4. 
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Discussion 

The  approximation  to  the  posterior  mean  and  variance  of  the 
ability  parameter  proposed  here  is  relatively  simple  to  compute 
once  the  posterior  mode  and  approximate  covariance  matrix  of  the 
item  parameters  are  available.  Other  approximations ,  which  do 
not  appear  as  readily  adaptable  to  ability  estimation,  yet  deserve 
further  study,  include  those  by  Leonard  (1982)  and  Tierney  and 
Kadane  (1986).  We  now  briefly  examine  the  computational  require¬ 
ments  of  these  approximations  for  evaluating  the  posterior  mean 
9  of  an  individual  with  response  x  who  appears  subsequent  to  the 
calibration  based  on  n  individuals  with  data  Y. 

For  the  Tierney  and  Kadane  approximation  define 

*u>  -  Jifji  p(yiji0i'£j,p(9i)d0i'  (22) 

L(  E,  )  =  m"1  log  n  (  £  )  t’(  O  .  (  23  ) 

t .  nd 

L*U)  *  m"llogE(  6  |X,U  n  (£)  lU  >  ,  (24) 

T 

where  n  ' f )  is  the  prior  for  C ,  m  =  n ♦ 1 ,  9  ,  -  A  and  (y  ,  ,  .  .  .  y  ,  ,  ) 

■  ~  K  -  n  + 1  7n*l,l  7n+l,k 

=  x  so  that  the  posterior  expectation  of  9  can  bt  expressed  as 

f 


E(9 | y  ,x  ) 


(  25  ) 
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A  A 

Suppose  and  are  the  modes  of  L  and  L*  and  Z1  and  Z*  are 

~  ~  ~  A  ~  A 

minus  the  inverse  Hessians  of  L  and  L*  evaluated  at  £'and 

respectively.  Under  regularity condit ions ,  the  posterior 
expectation  may  be  approximated,  according  to  Tierney  and  Kadane 
by 

|  Z  *  |  b  a  a 

g-  =  — 2 -  exp{m(L*(£*)  -  L(?')]}  •  (26) 

,  L  ~  ~ 

|  Z'  I 

One  of  the  requirements  is  that  E(0|x,  £)  be  a  positive  valued 
function  of  £  .  But  as  pointed  out  by  R.D.  Bock  (personal 
communicat ion ,  April  29,  1986)  since  the  origin  of  9  is  arbitrary 

we  may  take  it  to  be  a  large  positive  number,  such  as  5,  and 
if  necessary  truncate  the  prior  of  0  to  make  E(0|x,£)  positive. 

A 

Although  the  computation  of  (  .,  '  ,  Z  ')  can  be  performed  using 
the  EM  algorithm  as  shown  in  Tsutakawa  and  Lin  (1986),  the  compu- 

A 

tation  of  can  be  a  major  problem,  since,  unlike  the  compu- 

~  A 

tation  for  E,  '  ,  the  maximization  of  L*(£)  requries  working  with 
all  item  parameters  simultaneously. 

For  the  Leonard  approximation,  we  begin  with  the  joint 
posterior  of  (0,£,, 

p  (  9  •  5.  i  5 }  ■  P(  x  |  0  ,  f,  )P  (  0  )P(  £  I  y  )  (27) 

A 

For  fixed  9,  let  F  be  the  mode  of  f  and  R  the  negative  inverse 

A 

Hessian  of  the  log  of  (27)  with  respect  to  £,  evaulatedat  £  . 

Then,  under  regularity  conditions,  the  marginal  posterior  pdf 
of  9  is  approximated  by 


Ability  Estimation 


A 

P<e|x.X)  =  (  2tt  ) q/ 2  p  (  9  ,  £q  I  x  ,  y  )  /  !  R  0  i  ' '  .  (2  Hi 

where  q  is  the  dimension  of  Following  Leonard  and  Novick  i  .  8  > 
the  posterior  moments  of  0  may  be  computed  from  (28)  using 
numerical  integration. 

A 

In  practice  the  major  difficulty  would  be  in  computing 
The  EM  algorithm  can  again  be  used,  with  some  modification  t 

the  procedure  in  Tsutakawa  and  Lin  (1986).  However  since  r 

A  A 

computation  of  each  £  will  require  as  much  effort  as  comput  m  i 

y 

and  a  fair  number  of  (£  ,  |  R  j  )  pairs  will  be  needed  tor  eacp, 

~o  ~0 

individual,  without  some  modification,  Leonard's  approximat:  x. 
cannot  be  recommended  for  routine  use. 

In  practice  the  simplicity  of  our  approxima i ton  is  obtained 
by  separating  the  item  parameter  and  ability  estimation. 

Although  there  is  some  loss  of  information  from  not  using  all 
current  data  successively  update  the  posterior  distribution 
of  the  item  parameters,  (i)  the  extensive  computation  is 
completed  at  the  calibration  stage  and  (ii)  ability  measurement 
is  done  uniformly  and  without  excessive  computation  for  subsequent 


examinees . 
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